Mobility Unified Reporting System Administration and Management


Mobility Unified Reporting System Administration and Management
 
This chapter provides information on administering and managing the MUR application.
This chapter describes the following topics:
Important: Please note that the terminologies “starbi”, “inPilot” and “mur” used throughout this guide mean the same.
Launching the MUR GUI
It is recommended to use either Internet Explorer (v 7.0+) or Mozilla Firefox (v 3.0.10+) browser for launching the MUR interface.
Note that:
To launch the MUR interface:
1.
http://<MUR-server-hostname or IP address>:<apache port>
For example, http://10.4.5.2:8080
2.
Enter your user name and password, and then click Log In. The user name must be an alpha and/or numeric string of 3 through 16 characters in length. The only special character that a user name can include is underscore (_).
Important: At first log on, the users are expected to enter admin as the input for the Username and Password fields.
The password must meet the following criteria:
The only account created after the initial set-up is admin / admin and it has Administrator privileges.
Once logged-in, the user’s Dashboard will be displayed with reports if already configured (the displayed reports are specific to each user account).
Important: At first log on, the users will see an empty Dashboard. The necessary data should be populated and required parameters should be configured for report generation.
The user name is always displayed on the right-up corner of the page until the user logs out of the application.
Administration
This section provides information on how to administer and manage the MUR application.
Managing User Accounts
The MUR application provides two levels of access privileges:
Important: Only administrator with admin name can create user accounts.
Please note the following limitations with respect to user permissions and privileges:
All MUR administrators have access to USERS and GROUPS menu in the Admin tab available on the MUR GUI.
Administrator with admin user name will have the rights to modify and delete all the MUR users’ accounts. Only users with admin user name can modify its own password. Only admin user will be able to delete any administrator or operator user accounts.
Administrator other than users with admin user name will have rights to delete the MUR users except admin user and modify user accounts except their passwords.
For more details, see the Cisco Mobility Unified Reporting System Online Help documentation.
Managing Gateways
The MUR application supports configuring multiple gateways for which reports can be customized and generated. Gateways are the chassis from which EDR and bulkstat files are fetched to the reporting server.
Important: Users with administrative privilege can only add and manage gateways.
When a gateway is added through the GUI, a directory by the name of the gateway is created in the <mur_install_dir>/starbi/server/data directory.
The gateway directory structure looks like the following:
<data directory>
|
|--> <Gateway name>
|
|--> edr
The MUR application expects the EDR files in the directories that are created when adding the gateway.
The MUR application supports the distributed model to allow the deployment which enables network wide view or work load balancing. Newly introduced component, Remote Data Processor (RDP), plays the role of pre-processing the input files from gateways. One or more RDPs, installed separately on remote machines can be registered to a master MUR system and one RDP can process files from one or more gateways. The role of MUR system in such deployments is mostly for report generation, report viewing, RDP management and optionally data processing.
The RDP parses the raw data or EDR files from GGSNs and periodically forwards it to the registered master MUR application through SFTP for report generation. For information on how to configure the RDPs, see the Mobility Unified Reporting System Online Help documentation.
Important: The gateways can be added on Remote Data Processor (RDP). For adding gateway on particular RDP corresponding RDP's region should be selected. RDP region is available when RDP is added. For information on configuring the RDPs, see Mobility Unified Reporting System Online Help documentation.
In 12.0 and earlier releases, each of the registered RDPs were considered to be a new region. RDP region can be a child of the root of the MUR system (NOC) or can be the child of another region. However, all the gateways associated with a RDP will always be the children of RDP region.
Whenever an RDP is configured, internally MUR used to create corresponding region for the same. However, with the introduction of scalable MUR in 12.2 release, one gateway's files will be processed by two or multiple RDPs. In that case, RDP does not stand as a region. Hence, the reports would be required across all the RDPs under one particular region.
Important: Whenever an MUR is upgraded from an older version to 12.2, the logical regions for the MUR gets void and the user will not be able to see any gateways under the DPI/Bulkstats/CF/KPI tab. In this scenario, the user should add that gateway under NOC.
Managing Archive Directory
To use the Offline Subscriber Search feature seamlessly, you must organize the archive directory date-wise. For information on this feature, see the Mobility Unified Reporting Online Help documentation.
MUR organizes the archive directory such that the directory structure looks like the following:
<Archive Directory>
  <Gateway Name>
     <Reporting Name>
        <Date YYYYMMDD>
           <Archived Files>
              <Other>
           <Archived Files>
For the files that are not satisfying the required EDR file name format, MUR stores the files in the Other directory.
Configuring Logging
The MUR application facilitates logging to trace and debug problems identified within the reporting system.
Important: Users with administrative privilege can only manage logging.
Configuring Purging Feature
The MUR application supports purging any kind of aggregated data like half-hourly, daily, weekly, monthly, etc. This also supports purging of weekly summary table, monthly top N table, audit logs, etc. While configuring the purging feature, the MUR provides the flexibility to end-user to configure half-hourly, daily, weekly and monthly report viewing duration so that the historical reports can be viewed even at a lowest granularity level.
Important: It is recommended that you enable backup feature and take one complete successful snapshot of backup before enabling purging feature. If the backup feature is disabled then enabling purging will cause removal of data without waiting for it to be backed up.
Important: The backup snapshot can be identified as complete only if the snapshot directory of format snapshot_<date>_<time stamp>_<version>, created under configured backup path does not have the prefix backup.prog. The backup.prog prefix indicates that the backup is in progress.
MUR uses a python script, purge_db.py, to accomplish this task. For more information on the script, refer to the Using the Purging Script section.
Important: Users with administrative privileges can only manage the purging configuration.
Important: The purging configurations are recommended to be a one-time process and should not be changed frequently.
To configure data / file purging through the GUI, see the Mobility Unified Reporting System Online Help documentation.
Important: In case of distributed model of MUR, data purging can be done only at the master MUR and file purging can be performed at per RDP level.
Important: For the MUR software with version 10.0.72 and lesser, you must manually purge the archived files. For the MUR software version 10.0.72 and later, you can use the purging script to automate the process.
Configuring Backup Functionality
To avoid data loss due to hardware failure and/or software crash, MUR supports periodical backup and recovery of its database. The backup is actually the snapshot of data tables or meta data on the day when the backup is taken.
In case of hierarchical deployment, when backup feature is enabled, backup of RDP is also taken as per user configured period.
Backup of RDP contains only the metadata i.e. the configuration information and does not consume high disk space. Also this backup snapshot which is taken on RDP node is immediately transferred to Master MUR using SFTP.
The Master MUR moves the snapshot of RDP backup to the backup path configured on RDP. In short, RDP backup snapshot is also stored on user configured backup path.
Important: Please note that the backup and recovery processes are applicable only for MUR database and not for files that are archived.
Please consider the following points while taking backup of the master MUR database.
The backup snapshot can be identified as complete only if the snapshot directory of format snapshot_<date>_<time stamp>_<version>, created under configured backup path does not have the prefix backup.prog. The backup.prog prefix indicates that the backup is in progress.
Configuring Recovery Functionality
To recover the backed up data, use the snapshot recovery script that finds the latest available snapshot amongst all the snapshots under configured path. If you want to recover specific snapshot then move only that snapshot to some other path and provide this new path as a parameter to this script.
Important: In case of hierarchical deployment, recovery of master MUR and RDP should be done separately. After recovering and starting RDP, it will start serving the master to which it is attached.
Please note the following key points while recovering the database.
The recovery script provides an option to specify if recovery of data is required for RDP.
For recovery of RDP, backed up snapshot should be copied on a local path or it should be available on path that is accessible to RDP.
For the master MUR, use the following command to recover the data:
./recover.sh -path <directory path containing data snapshots>
For RDP, use the following command to recover the data:
./recover.sh -path <directory path containing data snapshots> -r <RDP_Name>
The option -r <RDP_Name> denotes the name of RDP for which data should be recovered.
The snapshot of RDP is a tar file and not a directory as in the case of master MUR. The following is a sample of RDP backup tar file naming convention.
RDP_<RDP_Name>_snapshot_<Date>_<timestamp>_<version>.tar
Configuring Offline Mode
MUR can be used in “Offline Subscriber Search Only” mode. In this mode, MUR will not parse incoming EDR files, so no online reports will be generated. It will move the incoming EDR files to archive directory, so that you can avail “Offline Subscriber Search Reports” only. By default, this mode is turned OFF. To run MUR in Offline Mode, please see the configuration parameters OFFLINE_MODE and OFFLINE_SEARCH_PROCESS_COUNT in Managing System Configurations > Config Parameters in the Mobility Unified Reporting System Online Help documentation.
Operations and Management
This section provides information on the scripts that can be used to manage the MUR components and the reports.
 
Using the Maintenance Utility
A shell script utility called serv is included with MUR in the <MUR_install_dir>/starbi/bin directory.
This serv script can be used to manage the following MUR processes:
This utility can report the status of the MUR processes on the system or it can be used to stop the MUR process.
Following are the options available with the serv script:
./serv { psmonitor | scheduler | postgres | apache | notif_server | parser | cache_server } [ start | stop | status ]
For example, if you want to start only the PSMON, then enter the following command:
./serv start psmonitor
or
./serv psmonitor start
Important: If you stop the MUR process, make sure that PSMON is not running. Otherwise PSMON will restart the MUR application.
The following is a sample output of the serv status command:
---------------------------------------------------
--------------- MUR Process Status ------------
 PID            Process                  Status
---------------------------------------------------
4245          Process Monitor           Running
4256          Scheduling server        Running
4267          Postgres Server         Running
4289          Apache Server           Running
3249          Notif Server           Running
3243          Parser Server           Running
2430          Cache Server           Running
---------------------------------------------------
Using the PSMON Script
PSMON is a perl script that is used to monitor the Scheduling Server, Postgres Server, and Apache Server processes. This script can start or stop the processes based on certain thresholds specified in the MUR configuration file. The PSMON respawns any dead processes using the set of rules defined in the configuration file.
This script can also optionally send notifications to users via e-mail.
Generating Reports in Excel Format
To generate the reports in excel format, execute the following script from the <MUR_install_dir>/starbi/bin directory.
./get_excel_report.sh -day <date for report generation> -f <path where report should be stored> -filter <filter for the report>
The script takes three parameters, the date for which report is to be generated, the path where generated report is to be stored, and the filter for the reports. The date must be in mm-dd-yyyy format only, and the filter can be based on Type Allocation Code (TAC) or Access Point Name (APN).
Using the unanonymize_msisdn.sh Script
MUR reports the subscribers data like Mobile Station Integrated Network (MSISDN) in the encrypted format both in the GUI and Excel file. Decryption functionality is ONLY supported on CLI through the use of unanonymize_msisdn.sh script.
This shell script utility will check for user’s privilege before decrypting the MSISDNs. It will prompt for the GUI administrator password.
Usage of unanonymize_msisdn.sh Script:
Important: Please note that this script must be run ONLY in bash shell.
./unanonymize_msisdn.sh -u <username> -f <input file> -d <output path>
To decrypt the MSISDN(s), perform the following steps:
1.
2.
3.
Provide this text file as an input to the unanonymize_msisdn.sh script.
Important: Please note that the users require GUI administrator credentials to access this utility.
Resetting GUI Administrator User Password
In case the Administrator user forgets the password, a set_admin_password.py script is used to reset the password. This script is located at the <MUR_install_dir>/starbi/server/scripts directory.
To reset the Administrator user’s password, perform the following steps.
Step 1
#source server/env.properties
#export PYTHONPATH
#export LD_LIBRARY_PATH
Step 2
#python2.6.1/bin/python server/scripts/set_admin_password.py
The script will update MUR database with Administrator user password as admin.
Using the generate_dns_mapp_sql.sh Script
To generate the DNS mapping for the specified list of IP addresses, execute the following script from the <MUR_install_dir>/starbi/bin directory:
./generate_dns_mapp_sql.sh <input file for IP> <output file where mapping should be stored>
This script is used to perform Internet DNS lookup of the specified IP addresses. It uses the ‘nslookup’ system administration command to find the DNS name of the specified IP. Please note that the machine must be connected to Internet for successful execution.
Generating Unknown URL Files
For CF reporting, MUR should parse CF-EDRs and generate the unknown/unrated URL database. This database will be pulled periodically by WEM and subsequently deliver to Rulespace. The unknown URL files can either be time based or count based.
To generate the unknown URL files, execute the following script from the <MUR_install_dir>/server/scripts/cfedr directory:
./gen_unknown_url.py
Important: Please note that up to a maximum of 85,000 Unknown URLs can be present in each file.
Using the getSupportDetails Script
In the event additional troubleshooting assistance is required, debugging information can be collected using a script called getSupportDetails.pl. This script collects different log files and captures the output of certain system commands that aid in troubleshooting issues. This script is packaged with MUR in the <MUR_install_dir>/starbi/tools/supportdetails/ directory.
This script refers to an XML file to get the list of logs. This XML file resides in the same directory as the script. Once executed, the script retrieves the contents of logs, files, folders, and output of certain commands and prepares a zipped file (MURsupportDetails.tar.gz), by default it is placed in /tmp/log directory.
Requirements
Perl 5.8.5 and above is required for running the script.
Apart from standard Perl modules (which are included in default installation of Perl), some additional modules are required for running the script. The list is as follows:
These modules are installed by default by the product. Please ensure that the above mentioned modules are installed when using a different installation of Perl.
To run the script, change to the directory path where the script is present and type:
./getSupportDetails.pl [--level=...] [--xmlfile=...] [--help]
For example, ./getSupportDetails.pl --level=4 --xmlfile=/tmp/getSupportDetails.xml
Supported Levels
The logs that can be collected for different levels are as follows:
netstat -an
ifconfig -a
df -k
etc..
ipcs
ps -eaf
etc..
Using the Purging Script
The python script, purge_db.py, handles both data purging and archived file purging. This script is packaged with MUR in the <MUR_install_dir>/starbi/server/scripts/utils/ directory.
This script runs daily at the end of the day, picks up the relevant tables, and then purges either data or archived files based on the configurations.
In case of data purging, the script picks up the relevant tables and purges them.
In case of file purging, the script purges the files only if the archived files are organized date-wise for each of the reportings like FLOW-EDR, HTTP-EDR, CF-EDR, and BS. For example, EDR file for 24th September, 2010 is stored in the archive/<gw>/flowedr/20100924 directory.
Server Script Parameters
The number of files being processed during each parsing interval for HTTP and non-HTTP EDRs can be controlled using the following parameters:
These parameters are defined in System menu under File parsing configs option available on the GUI.
With the above default configuration, if the number of files being accumulated are less than 25 and not in multiples of 5, then MUR spawns one more process to parse the remaining files.
Troubleshooting MUR
This section provides information on how to resolve situations you might encounter with using MUR software. This section provides problem definitions, their likely cause(s), and solutions.
On the Tools menu, click Clear Private Data.
Select Cache check box.
Click Clear Private Data Now.
On the Tools menu, click Internet Options.
Click Delete.
Select Temporary Internet files check box.
Click Delete.
Important: The Firefox version supported for MUR is 3.0.10 and later. For Internet Explorer, it is 7.0 and later.
Check if the prerequisites described in the Configuring Bulkstats Schemas section of the Configuring Chassis for MUR chapter are met.
On the ADMIN menu, check the bulkstats audit under AUDIT. The audit should indicate whether the bulkstats files are being parsed or not. For more information, refer to the Cisco Mobility Unified Reporting System Online Help documentation.
For more information, refer to the Cisco Mobility Unified Reporting System Online Help documentation.
For more information, refer to the Cisco Mobility Unified Reporting System Online Help documentation.
The entity pushing the files to MUR, for example, L-ESS should be added to MUR user group. For details, refer to the Managing Mobility Unified Reporting System Installation chapter of this guide.
If user is not able to configure bulkstats schema through Add Schema configuration screen that appears by selecting ADMIN > Bulkstats menu.
Check if the prerequisites described in the Configuring Bulkstats Schemas Using GUI section of the Configuring Chassis for MUR chapter are met.
Check if the SSH Username in the Add Bulkstats schema configuration screen is specified correctly. This user name is used to connect to gateway via SSH for schema configuration.
For information on how to configure the schemas, refer to the Configuring Bulkstats Schemas Using GUI section. Also, see the Cisco Mobility Unified Reporting System Online Help documentation.
Check if Username specified in the Add Bulkstats schema configuration screen is present in MUR group on the MUR server.
useradd <new user name>
usermod -G <MUR Group> <user name>
Check if Destination specified in the Add Bulkstats schema configuration screen is correct or not.
Restart apache server by executing the following command from the <mur_install_dir>/starbi/bin directory and try again to configure the schema.
./serv status shows Postgres processes as NOT RUNNING
The shared memory configuration in the /etc/system directory might not be correct.
Check if "shmmax" has been appropriately configured in the /etc/system directory (for Solaris users) or /etc/sysctl.conf directory (for Linux users). It should be set to 2684354560 (2.5GB). Reboot the system after making the changes to this file.
Check if the following variables in the sshd_config file present in the /etc/ssh directory are set appropriately.
Click ADMIN tab from the MUR GUI.
Click Edit for the gateway for which the EDR files are not getting parsed.
Check if there are any pending files in the RDP's /starbi/server/sql_export_data/ directory.
If there are any pending files, check if the following log is shown in the starbi_server_devel file located in the /starbi/logs/server/ directory.
Log on to the RDP machine. Switch the user as RDP admin. If you have set RDP admin as myrdp during RDP installation, then execute the following command su - myrdp.
Enter yes and quit the SFTP session.
Check the files again in the starbi/server/sql_export_data directory to identify if they are present now. Then, check the reports after a while.
If the reports are still not seen, check the master configuration in System menu. Check if the master login and password are specified. Ensure that master admin password has been set using password command on the master.
./serv status shows cache server is not running.
 
 
 

Cisco Systems Inc.
Tel: 408-526-4000
Fax: 408-527-0883